Skip to content

perf(huddle): enable bit-exact streaming TTS by default - #6334

Closed
johnmatthewtennant wants to merge 3 commits into
jtennant/harden-pocket-tts-streamingfrom
jtennant/enable-pocket-tts-streaming
Closed

perf(huddle): enable bit-exact streaming TTS by default#6334
johnmatthewtennant wants to merge 3 commits into
jtennant/harden-pocket-tts-streamingfrom
jtennant/enable-pocket-tts-streaming

Conversation

@johnmatthewtennant

Copy link
Copy Markdown
Contributor

Context

PR #5671 reduced Pocket TTS time-to-first-audio by streaming stateful Mimi decoder output, but kept the path disabled by default pending exactness, cancellation, failure, and listening validation.

The 12-frame decoder boundary produces the same samples as batch decoding. The streaming path now also preserves queued speech across inference failures, fades partial tails, and keeps cancellation from flushing retained audio.

Summary

Enable Pocket TTS streaming by default at the Mimi decoder's native 12-frame boundary. Operators can restore batch synthesis with BUZZ_TTS_STREAMING=0 or BUZZ_TTS_STREAMING=false.

Changes

  • Defaults streaming_emit_frames() to 12 instead of requiring BUZZ_TTS_STREAMING=1.
  • Accepts the repository's standard 0 and false values for operational rollback.
  • Preserves BUZZ_TTS_EMIT_FRAMES for explicit latency experiments.
  • Tests default, rollback, explicit, and invalid environment resolution without mutating process-global environment variables.
  • Tightens the real Pocket model integration test to require exact equality at 12 frames and print the measured maximum sample difference.

Related issue

No issue found. Promotion follow-up to #5671.

Testing

The installed April Pocket model produced delta_frames=12: max|diff|=0.000000 when the same real latent sequence was decoded through the batch and stateful streaming paths. The integration test now requires exact equality rather than accepting a tolerance.

The default and both rollback values are covered through the pure environment resolver.

A real huddle ear and barge-in soak pass is still required before this draft is marked ready. The current environment can exercise the model and native pipeline deterministically, but cannot independently judge audible quality or conduct a two-party huddle.

Screenshots

Not applicable. This changes the native audio pipeline and has no visual UI change.

Reviewer-reproducible examples

From a fresh checkout with the Pocket model installed at ~/.buzz/models/pocket-tts:

. ./bin/activate-hermit
BUZZ_POCKET_TEST_MODEL_DIR="$HOME/.buzz/models/pocket-tts" \
  cargo test -p buzz-voice incremental_stateful_decode_matches_batch_decode \
  -- --ignored --nocapture

Observed output excerpt:

delta_frames=6: max|diff|=0.211313 rms_err=0.009880 snr_db=21.8
delta_frames=4: max|diff|=0.279904 rms_err=0.011155 snr_db=20.7
delta_frames=2: max|diff|=0.228128 rms_err=0.011197 snr_db=20.7
delta_frames=1: max|diff|=0.231133 rms_err=0.011362 snr_db=20.6
delta_frames=12: max|diff|=0.000000
test pocket::pocket_april::tests::incremental_stateful_decode_matches_batch_decode ... ok

For the remaining manual gate, run this branch with no BUZZ_TTS_STREAMING environment variable, enable Pocket TTS, and request a long multi-sentence response in a real huddle. Confirm speech begins during synthesis, remains continuous, and stops cleanly for PTT, barge-in, and speaker removal. Repeat with BUZZ_TTS_STREAMING=false and confirm batch playback remains available as a rollback.

@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/harden-pocket-tts-streaming branch from df5c133 to b348a00 Compare August 19, 2026 18:33
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/enable-pocket-tts-streaming branch from 06c90fe to 49b4aca Compare August 19, 2026 18:33
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/harden-pocket-tts-streaming branch from b348a00 to 2b01e7f Compare August 19, 2026 18:50
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/enable-pocket-tts-streaming branch from 49b4aca to 4896e15 Compare August 19, 2026 18:50
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/harden-pocket-tts-streaming branch from 2b01e7f to f445345 Compare August 19, 2026 18:51
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/enable-pocket-tts-streaming branch from 4896e15 to 0bdc626 Compare August 19, 2026 18:51
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/harden-pocket-tts-streaming branch from f445345 to 0a4010a Compare August 19, 2026 19:03
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/enable-pocket-tts-streaming branch from 0bdc626 to d4d0f0c Compare August 19, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant